Methods for distance estimation to regions in a scene during image capture can be divided into two main approaches: active and passive. Active approaches involve additional energy sources such as illumination sources to determine the distance to objects in the scene. These additional energy sources substantially increase the energy required for capture. In contrast, passive approaches determine the distance to objects in the scene by analysis of changes of viewpoint or focus without using additional illumination sources and as such are more energy efficient.
In a first class of passive depth estimation methods, multiple viewpoints are obtained by capturing multiple images as in stereovision. Alternately, multiple viewpoints can be captured simultaneously by positioning a lenticular array over an image sensor as described in U.S. patent application Ser. No. 12/040,274, entitled “Sensor with Multi-perspective Image Capture.” In another method, images with different perspectives can be captured by alternately blocking half of the aperture of the lens of the image capture device as disclosed in U.S. patent application Ser. No. 12/259,348, entitled “Split Aperture Capture of Rangemap for 3D Imaging.” In each of these methods, depth is estimated by comparing the relative locations of objects in the multiple images and determining the distance to the objects by triangulation.
In a second class of passive depth acquisition methods, depth is estimated by comparing the focus quality for objects in multiple images that were captured from a single viewpoint using multiple focus settings where the lens is focused at different depths. However, these first and second classes of passive depth estimation methods all require multiple images to be captured and compared to estimate depth, thus increasing the computational complexity and increasing the processing time required.
A split color filter system is another version of triangulation that can be used to estimate depth in an image of a scene. In a split color filter system, a split color filter is inserted into the optical path of the lens at the aperture position thereby creating 2 optical paths with different perspectives. The split color filter is constructed so that the filter area is divided into at least two different areas with different colors (typically red and blue) in each area. Two images are then captured simultaneously as a first image formed with light passing through one side of the filter is overlaid on top of a second image formed with light passing through the other side of the filter. Since the first and the second images have different colors, they can be differentiated in the overlaid image. Any defocused regions present in the image have an offset between the two colored images due to the different perspectives of the two optical paths, which then shows up as color fringes on either side of the object in the image. Changing the focus distance of the lens reduces or enlarges the color fringes in the image depending on the distance from the defocused region to the focus distance. When an object in the image is well focused, the color fringes disappear. When an object is defocused inside of the focal distance, the fringes are one color on one side of the object and the other color on the other side of the object in the image. When an object is defocused outside of the focal distance, the colors of the color fringes are reversed. Consequently, with this approach, one image taken with the split color filter delivers an image that can be analyzed to estimate the depth of objects in the scene based on the size and orientation of the color fringes on either side of the objects in the image.
A split color filter system for autofocus is described by Keiichi in the Japanese Patent Application 2001-174696 where a red and blue split color filter is used. Another autofocus system using a color filter with multiple apertures is presented in U.S. Patent Application Publication 2006/0092314. In this disclosure, a color filter with two or three different single colors (red, green, and blue) at the aperture creates two or three overlaid images of different colors (red, green, and blue) on the sensor. All these methods based on split color filters for auto-focus introduce an added complexity by altering the color of the images in the different optical paths for each color filter.
In U.S. Patent Application Publication 2008/0247670, a sparse depth map is built using edge analysis to render stereoscopic images. To enable all of these techniques to be used for image capture and stereoscopic image pair generation where color accuracy is important, this alteration in color of the image must be corrected during or after the image capture so that an image can be produced with accurate color makeup within the image.
A need exists for a method to generate stereoscopic image pairs in a scene from a single image capture with a single lens without a substantial loss of image quality or a substantial loss of color accuracy.